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Abstract 

Medical students consider anatomy, neurology, and traumatology as difficult study topics. A 
recent study showed that the daily quiz ‘Two Opportunities to Practice per day (TOPday)’ 
positively supported biomedical students in analyzing and solving biomechanical problems. 
The main purpose of this study was to investigate the effect of TOPday on self-confidence, 
enthusiasm, and test results for the topics anatomy, neurology and traumatology. Second-year 
medical students were enrolled in a four-week course on the human skeletal system at the 
Radboudumc (n = 799). They were randomized over three topic groups (anatomy, neurology, 
and traumatology) and received TOPday quizzes on every course day. At the end of the 
course students filled in a non-anonymous questionnaire. Students highly appreciated 
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TOPday (7.5±0.9) and this did not differ between groups (anatomy: 7.4±0.8; neurology: 
7.4±1.1; traumatology: 7.5±0.8; P = 0.68). Many students reported that TOPday increased 
their self-confidence (65% of the students) and enthusiasm (69% of the students) for their 
topic. However, test results of the students did not improve. A potential explanation for the 
latter result may relate to the different cognitive processes that are required to study anatomy, 
neurology, and traumatology compared to biomechanics. In conclusion, appreciation, 
self-confidence and enthusiasm were positively associated with TOPday, but test results were 
not. 

Keywords: Daily quiz, Fonnative test, Feedback, Digital environment, Repeated learning 

1. Introduction 

High-quality patient care is only feasible if physicians have received high-quality teaching 
during both their undergraduate and their residential years (Leach, 2001; Leach & Philibert, 
2006). Due to a growing body of knowledge, the rapid development of information and 
communication technology, lifelong learning has become essential for future physicians. As 
deep approaches to learning are a key element of lifelong learners, the learning environment 
should stimulate this type of learning (Baeten, Dochy, & Struyven, 2008). This means that 
students should be active participants in the process of learning and self-regulate their 
learning (Baeten et ah, 2008). For this self-regulation, feedback on the performance is 
essential (van Dinther, Dochy, & Segers, 2011). If feedback is constructive it can lead to an 
increasing self-efficacy that in return is related to better academic performance (Chemers, Hu, 
& Garcia, 2001). 

A digital learning environment can contribute to this kind of learning environment, whereby 
students are actively involved in their own education (Lameris, Hoenderop, Bindels, & 
Eijsvogels, 2015; Ruiz, Mintzer, & Leipzig, 2006; Tanck, Hannink, van Kuppeveld, Bolhuis, 
& Kooloos, 2013). Within such a learning enviromnent repeated testing like spaced education 
has been shown to be a valuable element (Karpicke & Roediger, 2008; Kerfoot, DeWolf, 
Masser, Church, & Federman, 2007; Roediger & Karpicke, 2006). Spaced education 
combines two principle psychological research findings, i.e., temporal distribution of learning 
(spacing effect) and retrieval practice (testing effect) (Kerfoot, 2010; Larsen, Butler, & 
Roediger, 2008; Pashler, Rohrer, Cepeda, & Carpenter, 2007; Sisti, Glass, & Shors, 2007). 

A promising method to combine a digital environment and spaced education is described by 
Tanck et al. (2014), who developed ‘Two Opportunities to Practice per Day’ (TOPday). 
TOPday is a digital daily quiz about biomechanics with immediate feedback. TOPday 
motivated biomedical students to be actively involved with this topic and increased their 
enthusiasm as well as their self-confidence for biomechanics. In addition, their test results for 
biomechanics were significantly higher compared to students who did not use the TOPday 
(Tanck et al., 2013; Tanck et al., 2014). 

Although TOPday may be an effective tool to support biomedical students’ problem-driving 
education, TOPday has not been applied or investigated in other disciplines of the medical 
curriculum. At the Radboud university medical center, we offer medical students in their 
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second year a four-week course about the human musculoskeletal system. This course 
contains many new concepts and complex terminologies regarding anatomy, neurology, and 
traumatology. As these topics are often considered difficult topics by students, we wanted to 
know whether TOPday could be applied in a knowledge driven environment and support the 
learning of students (Carley & Driscoll, 2001; Pandey & Zimitat, 2007; Schon, Hart, & 
Fernandez, 2002; Zinchuk, Flanagan, Tubridy, Miller, & McCullough, 2010). 

Therefore, the main purpose of this study was to investigate the effect of ‘Two Opportunities to 
Practice per day (TOPday)’ on self-confidence, enthusiasm, and test results for the topics 
anatomy, neurology, and traumatology. We hypothesized that students who use TOPday will 
have higher study results, are more enthusiastic and feel more self-confident about the topics 
compared to students who do not use TOPday. 

2. Methods 

2.1 Students 

This study was conducted in two cohorts (2012 and 2013) of second-year medical students (n 
= 799) who were enrolled in a four-week course on the human musculoskeletal system at the 
Radboudumc in Nijmegen, The Netherlands. Students could voluntarily attend lectures and 
were asked to do self-study-assignments covering anatomy, neurology, and traumatology. 
After each self-study-assignment, students could attend an interactive lecture where 
difficulties from the assignment and other questions of the students were discussed with the 
lecturer. After the four-week course, the students had to do a written examination that 
included the topics anatomy, neurology, and traumatology. 

2.2 TOPday 

During the course, medical students were encouraged to frequently study the different topics 
of the human musculoskeletal system via self-study assignments. TOPday was developed to 
aid and stimulate students to repeatedly study the subject matter. Medical students were 
randomly divided over three topic groups (anatomy, neurology, and traumatology). During 
the first three weeks of the course, the students received one TOPday quiz per day 
(Monday-Friday) about their topic via e-mail. The TOPday quiz consisted of one-best-answer 
items with increasing difficulty over the course. The questions were written by expert 
anatomists, neurologists, and trauma surgeons involved in the course and required less than 
10 minutes to answer. After answering the questions online, students received feedback 
immediately (Figure 1). The rationale for this approach was to increase the students’ 
motivation for repetitive learning and to give them an opportunity for self-assessment. In total, 
15 TOPday quizzes per topic were sent to the students. At the end of the course, students 
were invited to fill in a questionnaire after which they performed the final examination. The 
three topics accounted for 36% of the total test score, where traumatology accounted for 14% 
and anatomy and neurology for 11% each. 


103 


www.macrothink.org/jei 




Macrothink 

Institute™ 


Journal of Educational Issues 

ISSN 2377-2263 
2016, Vol. 2, No. 2 


Example: Question 14 

The capitate bone is found within the distal row of carpal bones. What bone structure from the 
proximal row articulates with the capitate bone most prominently? 

A. Triquetral bone 

B. Pisiform bone 

C. Lunate bone 

D. Scaphoid bone 

Motivation of the correct answer (C) : 

The lunate bone lies proximally from the capitate bone. The proximal end of the capitate bone 
has a convex shape, whereas the lunate bone has concave shape. The joint between both bone 
structures has a so called ball and socket joint. The capitate bone articulates also with the 
scaphoid bone, but the articulation is more prominent by the lunate bone. 


Figure 1. Example of a TOPday question on Anatomy (translated from Dutch) 


2.3 Ethical Considerations 

A week before the course, students were informed about the procedures and purpose of 
TOPday via e-mail. During the introductory lecture, students were informed about the 
procedure. It was stressed that participation was voluntarily and that they could unsubscribe 
from TOPday at any time without any consequences. In addition, students were assured that 
completing the non-anonymous questionnaire would not affect or influence their study grades. 
Privacy and confidentiality were guaranteed and results of this study were only accessible for 
the researchers involved. The questionnaire was linked to a student number, which was 
subsequently linked to the results of the final examination. Ethical principles of the World 
Medical Association Declaration of Helsinki were taken into account and students gave 
written consent to use the data for research purposes. 

2.4 Outcome Measures and Statistical Analysis 

After the final TOPday question, students were asked to complete a non-anonymous 
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questionnaire containing 15 items, focused on motivation, appreciation, and organization/ 
logistics of TOPday. Students could also provide comments about the organization and 
content of TOPday. The questionnaire was made by experts in the field of study evaluation 
questionnaires. The reliability of the questionnaire was evaluated via Cronbach’s Alpha. 
Students who completed the questionnaire as well as the final examination were included in 
the data analysis. Students who exchanged and completed TOPday questions of other topics 
than their own were excluded from the primary data analysis. 

The following parameters were used for the statistical analysis: 

(1) Questionnaire outcomes: 

a. TOPday participation rate; number of TOPday questions completed (0 to 5, 6 to 10, 11 to 
15 questions). This was asked for all topics to check for any exchange of questions between 
students. The three answer options were further categorized into low (0 to 5 questions, 
group A) and high (6 to 15 questions, group B) participation. 

b. Increase (yes/no) in enthusiasm and self-confidence for anatomy, neurology, or 
traumatology, due to TOPday. 

c. The students’ appreciation for TOPday (scale 1 [very bad] to 10 [excellent]). 

d. Organization of TOPday (1 [bad], 2 [sufficient], or 3 [good]). 

e. Content of TOPday quizzes (1 [irrelevant] or 2 [relevant]). 

f. Quality of the feedback of TOPday quizzes (1 [unclear] or 2 [clear]). 

(2) Grades examination: 

a. Total test score for the final examination (scale 0-10). 

b. Sub-scores for anatomy, neurology and traumatology (scale 0-10). 

To detennine whether TOPday quizzes were associated with an increase in test score per topic, 
the number of TOPday quizzes completed versus test results were analyzed using two-way 
ANOVA and Tukey post hoc tests with factors: 1) received topic; 2) participation rate (group A 
[low] or B [high]); and 3) interaction between received topic and participation rate. 
Self-confidence and enthusiasm were analyzed with descriptive statistics in the group of 
students who actively participated (B group). In the results, data is shown as mean and standard 
deviation (SD). Statistical significance was set at P < 0.05. 

3. Results 

3.1 Participation Rate and Student Population 

The questionnaire had a good reliability (Cronbach’s alpha 0.88). After the exclusion of 
students that did not participate in the final examination or questionnaire (n = 197), or 
students that completed multiple TOPday quizzes from other topics (n = 226), 376 students 
were included for the final statistical analyses. Of these 376 students, 72 (19%) had a low 
participation rate in TOPday (group A) and 304 (81%) had a high participation rate in 
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TOPday (group B). The number of males (31%) and females (69%) as well as TOPday 
participation rate did not differ between the three topics (P = 0.51 and P = 0.69, respectively; 
Table 1). 


Table 1. Characteristics and TOPday participation rate of 374 medical students categorized by 
topics anatomy, neurology, and traumatology 


Allocated Topic 

Anatomy 

Neurology 

Traumatology 

p-Value 

n 

n 115 

n 137 

n 124 

- 

Sex (males, n) 

31 (27%) 

43 (31%) 

42 (34%) 

0.51 

Participation Rate 

Group A (0-5 questions) 

22 (19%) 

29 (21%) 

21 (20%) 

0.69 

Group B (6-15 questions) 

93 (81%) 

108 (79%) 

103 (80%) 

Appreciation of TOPday 

Overall students’ appreciation 

7.4±0.8 

7.4±1.1 

7.5±0.8 

0.68 

Organization (good; n, %) 

84% 

85% 

83% 

0.86 

Content of TOPday (relevant; n, %) 

87% 

77% 

85% 

0.11 

Feedback of TOPday (clear; n, %) 

85% 

87% 

94% 

0.10 


3.2 Appreciation 

The students’ appreciation of TOPday did not differ between the three topics (Table 1). 
Within each topic, students of group B appreciated TOPday significantly higher compared to 
students of group A (anatomy: A 6.8±0.9 vs. B 7.5±0.8; neurology: A 6.9±1.5 vs. B 7.6±0.9; 
traumatology: A 7.1±1.0 vs. B 7.6±0.8; P < 0.01). The majority of the students were satisfied 
with the organization, content, and feedback of TOPday and this did not differ between the 
three topics (Table 1). In general, students commented that they liked TOPday, but found it 
disappointing that they received only one topic instead of all three. Another general remark of 
the students was that they appreciated the feedback provided after they answered the TOPday 
quiz. 

3.3 Self-Confidence and Enthusiasm 

The majority of the students (n = 244, 65%) who actively participated in TOPday, perceived 
an increase in self-confidence for their topic because of TOPday. This did not significantly 
differ between the topics anatomy (n = 67, 72%), neurology (n = 78, 72%), and traumatology 
(n = 81, 79%) (P = 0.46). Likewise, the majority of the students (n = 259, 69%) who actively 
participated in TOPday indicated that their enthusiasm for their topic increased because of 
TOPday. This did not significantly differ between the topics anatomy (n = 73, 78%), 
neurology (n = 83, 77%), and traumatology (n = 80, 78%) (P = 0.96). 
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3.4 Test Results 

Test results of the final examination did not differ between students who received TOPday 
quizzes on anatomy (7.0±0.9), neurology (7.2±0.9), or traumatology (7.0±1.0), P > 0.05. 
Within each topic, there were no significant differences in test results for the topics between 
group A and B (Figure 2 A). 


A. Sub scores of included students 


10 - 

9 - 

8 - 



n 22 93 29 108 21 103 


M Anatomy 

□ Neurology 

□ T raumatology 


B. Sub scores of excluded students 

10 - 

9 - 





A B A B 


Anatomy 

Neurology 

Traumatology 



A B 



n 36 51 41 30 28 40 


Figure 2. Sub score of the three topics (anatomy [black], neurology [dark grey], and 
traumatology [light grey]) grouped by the number of questions made (group A [0-5 questions], 
group B [6-15 questions]) for the A. included students and B. excluded students 

Note. The sub score for anatomy, neurology, and traumatology did not differ between group A 
and B for either one of the topics. Data is presented as mean ± SD. 
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3.5 Sub Analysis: Excluded Students due to Multiple TOP day Topics 

Since we excluded a substantial number of students (n = 226) who completed multiple 
TOPday quizzes from other topics, we performed a sub analysis on this group. The majority 
of the excluded students appreciated TOPday, perceived an increase in self-confidence, and 
were enthusiastic for the topic (Table 2). Test results of the final examination did not differ 
between students who received TOPday quizzes on anatomy (7.0±0.9), neurology (7.1±1.1), 
or traumatology (7.1±0.8), P > 0.05. Within each topic, there were no significant differences 
in test results for the topics between group A and B (Figure 2 B). Students who made TOPday 
quizzes from multiple topics had higher test results for anatomy compared to students who 
made TOPday quizzes about one topic (6.1±2.2 vs. 5.3±2.4, P = 0.002), but lower test results 
for traumatology (5.8±1.3 vs. 6.3±1.3, P = 0.002). Neurology (7.3±2.1 vs. 7.6±2.1, P = 0.14) 
or total test score (7.1±0.9 vs. 7.1±0.9, P = 0.70) did not differ between students with multiple 
TOPday quizzes and students with one TOPday quiz, respectively. 


Table 2. Characteristics, study results and TOPday participation rate of 226 medical students 
who were excluded due to the exchange of the topics anatomy, neurology, and traumatology 


Allocated Topic 

Anatomy 

Neurology 

Traumatology 

p-Value 

n 

n 87 

n 71 

n 68 

- 

Sex (males, n) 

35 (40%) 

26 (37%) 

25 (37%) 

0.87 

TOPday Participation Rate 

Group A (0-5 questions) 

36 (41%) 

41 (58%) 

28 (41%) 

0.07 

Group B (6-15 questions) 

51 (59%) 

30 (42%) 

40 (59%) 

TOPday 

Topic 

Group 

Topic* Group 

Students’ Appreciation 

Group A 

7.6±1.2 

7.1±0.8 

7.5±0.7 

0.05 

0.045 

0.33 

Group B 

7.8±1.0 

7.6±1.1 

7.5±0.9 

Increase self-confidence (n, %) 

41 (80%) 

21 (70%) 

31 (78%) 

- 

0.56 

- 

Increase enthusiasm (n, %) 

40 (78%) 

22 (73%) 

32 (80%) 

- 

0.79 

- 


4. Discussion 

This study showed that the majority of the medical students who participated and completed 
the questionnaire highly appreciated TOPday and that they were motivated to be actively 
involved with the subject matter on a voluntary basis. Most students actively participated and 
reported that they appreciated this approach by email, including the feedback. Participation in 
TOPday did not require a lot of time and effort, indicating easy accessibility to participate. A 
substantial number of students perceived an increase in self-confidence and enthusiasm for 
the subject matter because of TOPday. In contrast, TOPday did not improve the test results. 

Students indicated that they perceived TOPday quizzes as useful. This may relate to the 
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voluntary aspect and immediate feedback after TOPday quizzes. The observed increase in 
enthusiasm and appreciation for (digital) study aids is in line with other studies (Evans, Zeun, 
& Stanier, 2014; Palmen, Vorstenbosch, Tanck, & Kooloos, 2015). The rationale for this 
approach was to increase the students’ motivation for repetitive learning and to give them an 
opportunity for self-assessment, which fits in the recent changes of the medical curriculum at 
the Radboudumc. The recent changes place the student in the centre of their education by 
taking self-directed learning as starting-point. The students are responsible for their education 
and have to create a portfolio in which they reflect on their progression in medical education. 

In the previous work of Tanck et al. (2014), a positive association was found between 
TOPday and test results for biomechanics. Despite the fact that students in the present study 
were stimulated to repeatedly study the subject matter, a higher participation rate for TOPday 
was not significantly associated with a higher final examination score. A potential 
explanation for this observation may relate to different cognitive processes required to study 
anatomy, neurology, and traumatology compared to biomechanics (Krathwohl, 2002). At the 
Radboudumc, learning anatomy, neurology, and traumatology, mainly requires remembering 
and understanding cognitive skills, whereas biomechanics requires higher cognitive skills, 
such as applying and analyzing concepts (Krathwohl, 2002). Possibly the effect of additional 
practice through TOPday is more rewarding when the topic is relatively complex or the 
obtained knowledge can be applied in different situations, like in biomechanics, whereas 
TOPday is less rewarding in case of retrieval practice of facts. 

Previous studies indicated that the participation rate in formative quizzes was correlated to 
summative examination scores (Kibble, 2007; Palmen et al., 2015). The present study did not 
confirm this finding. Although students who completed TOPday quizzes from multiple topics 
performed better on anatomy, their test results for neurology or total test score did not differ 
compared to students who made TOPday quizzes for one topic. It should be noticed that it is 
rather difficult to find a causal relationship between TOPday and test results, since other 
competing learning activities ( e.g self-study assignments and lectures) were involved during 
the course as well (Roediger & Karpicke, 2006). The number of TOPday questions could be 
increased in the future, resulting in better coverage of the subject matter which might 
improve the test results of the students (Kibble, 2007; Palmen et al., 2015). However, 
increasing the number of questions requires that the students have to spend more time on 
TOPday, which might discourage students to participate with quizzes (Logan, Thompson, & 
Marshak, 2011; Palmen et al., 2015). Balancing the number of questions and time 
consumption for the students to complete the quiz is important. Students indicated they would 
have liked TOPday questions for all three of the topics. Some students therefore exchanged 
the different TOPday questions with each other. Although the students who did more than one 
topic were excluded from the analyses, there will remain uncertainty about the degree of 
honesty among students who filled in the questionnaire. 

There are other possibilities to stimulate students to actively recall their knowledge or to 
search for additional information on the subject matter, such as bla nk exercises or 
supplementary questions as formative testing methods (Timby & Smith, 2010). In addition, 
motivating students by providing them credit for the participation in quizzes, might also 
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enhance the learning efficiency (Kibble, 2007). Alternatively, the TOPday questions of this 
study could be modified towards higher levels on the cognitive process dimension 
(Krathwohl, 2002). Then, our hypothesis that TOPday is more effective to improve test 
results when higher cognitive process skills are required could actually be tested. The 
questions at the final examination should then be changed towards higher cognitive levels as 
well (Crowe, Dirks, & Wenderoth, 2008). Future research is warranted to detennine whether 
TOPday is more effective to improve test results when higher cognitive process (e.g., applied) 
skills are required than when lower (e.g., memory) skills are required to answer the questions. 

4.1 Methodological Considerations 

This study is inherent to several limitations. First, we were unable to automatically track 
whether students exchanged TOPday quizzes between the different topics. To be able to take 
this into account, we asked the students whether they exchanged TOPday quizzes with each 
other. A second limitation is that we sent the TOPday quizzes via email so that TOPday 
quizzes were always available. Therefore, it is possible that some students saved TOPday 
quizzes until later in the course and made them just before the final examination. 

5. Conclusion 

Students highly appreciated TOPday and were motivated to be actively involved with the 
subject matter for anatomy, neurology, and traumatology. Students who participated actively 
in TOPday, perceived an increase in self-confidence and enthusiasm for the subject matter 
because of TOPday. However, we did not observe an increase in their test results compared to 
students who not actively participated in TOPday. 
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